Respect default_args in DAG when its set to a "falsy" value#57853
Merged
amoghrajesh merged 14 commits intoapache:mainfrom Nov 17, 2025
Merged
Respect default_args in DAG when its set to a "falsy" value#57853amoghrajesh merged 14 commits intoapache:mainfrom
amoghrajesh merged 14 commits intoapache:mainfrom
Conversation
Member
|
The ser dag tests are failing, as we've removed some default falsy values. I think that's okay, but we need to examine those closely. |
ashb
reviewed
Nov 5, 2025
ashb
reviewed
Nov 5, 2025
airflow-core/tests/unit/serialization/test_dag_serialization.py
Outdated
Show resolved
Hide resolved
Contributor
Author
|
👁️ looking at the failures |
Contributor
Author
|
@ashb this one wasn't as easy as I thought. The PR is updated now, can you take a look? |
uranusjr
reviewed
Nov 6, 2025
Lee-W
reviewed
Nov 11, 2025
Member
|
rebasing for triggering the test |
Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
Contributor
Author
|
@Lee-W it worked haha! |
Lee-W
approved these changes
Nov 11, 2025
Member
Lee-W
left a comment
There was a problem hiding this comment.
Nice! Then others are nits from my end.
ashb
reviewed
Nov 11, 2025
Member
ashb
left a comment
There was a problem hiding this comment.
Is it worth also checking out works
uranusjr
reviewed
Nov 12, 2025
airflow-core/tests/unit/serialization/test_dag_serialization.py
Outdated
Show resolved
Hide resolved
Contributor
ephraimbuddy
left a comment
There was a problem hiding this comment.
Looks good. My only issue is on performance
ephraimbuddy
approved these changes
Nov 17, 2025
github-actions bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Nov 17, 2025
…ue (apache#57853) (cherry picked from commit c03fc79) Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
aaron-wolmutt
pushed a commit
to aaron-wolmutt/airflow
that referenced
this pull request
Nov 20, 2025
2 tasks
itayweb
pushed a commit
to itayweb/airflow
that referenced
this pull request
Dec 6, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #57792
Problem
When
retries=0was explicitly set in DAGdefault_args, the value was being ignored during serialization, causing tasks to fall back to thedefault_task_retriesconfiguration value instead of the explicitly set0.Example
Cause
The serialization logic excluded values that matched schema defaults (e.g.,
retries=0) from the serialized json even when they differed fromclient_defaults(e.g.,retries=3from config). During deser, whenretrieswas missing from the serialized task data it would result it to back toclient_defaults.tasks.retries(3), losing the explicitly set value (0).Fix
Updated the ser/deser logic to only exclude values that match both the schema default AND
client_defaults. If a value matches the schema default but differs fromclient_defaults, it is included because that means it was explicitly set.Additionally, added a generic method
get_operator_const_fields()to detect fields with"const": truein the schema (like_is_sensorand_is_mapped), which should always be excluded whenFalse, regardless ofclient_defaults.Testing
Before:
After:
Tested with DAG:
Result with no retries:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.